Search Results: "aron"

31 March 2017

Chris Lamb: Free software activities in March 2017

Here is my monthly update covering what I have been doing in the free software world (previous month):
Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed pre-compiled to end users. The motivation behind the Reproducible Builds effort is to permit verification that no flaws have been introduced either maliciously or accidentally during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised. I have generously been awarded a grant from the Core Infrastructure Initiative to fund my work in this area. This month I:
I also made the following changes to our tooling:
diffoscope

diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.

  • New features/optimisations:
    • Extract squashfs archive in one go rather than per-file, speeding up ISO comparison by ~10x.
    • Add support for .docx and .odt files via docx2txt & odt2txt. (#859056).
    • Add support for PGP files via pgpdump. (#859034).
    • Add support for comparing Pcap files. (#858867).
    • Compare GIF images using gifbuild. (#857610).
  • Bug fixes:
    • Ensure that we really are using ImageMagick and not the GraphicsMagick compatibility layer. (#857940).
    • Fix and add test for meaningless 1234-content metadata when introspecting archives. (#858223).
    • Fix detection of ISO9660 images processed with isohybrid.
    • Skip icc tests if the Debian-specific patch is not present. (#856447).
    • Support newer versions of cbfstool to avoid test failures. (#856446).
    • Update the progress bar prior to working to ensure filename is in sync.
  • Cleanups:
    • Use /usr/share/dpkg/pkg-info.mk over manual calls to dpkg-parsechangelog in debian/rules.
    • Ensure tests and the runtime environment can locate binaries in /usr/sbin (eg. tcpdump).

strip-nondeterminism

strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build.

  • Fix a possible endless loop while stripping .ar files due to trusting the file's own file size data. (#857975).
  • Add support for testing files we should reject and include the filename when evaluating fixtures.

buildinfo.debian.net

buildinfo.debian.net is my experiment into how to process, store and distribute .buildinfo files after the Debian archive software has processed them.

  • Add support for Format: 1.0. (#20).
  • Don't parse Format: header as the source package version. (#21).
  • Show the reproducible status of packages.


Debian


I submitted my platform for the 2017 Debian Project Leader Elections. This was subsequently covered on LWN and I have been participating in the discussions on the debian-vote mailing list since then.


Debian LTS

This month I have been paid to work 14.75 hours on Debian Long Term Support (LTS). In that time I did the following:
  • "Frontdesk" duties, triaging CVEs, etc.
  • Issued DLA 848-1 for the freetype font library fixing a denial of service vulnerability.
  • Issued DLA 851-1 for wget preventing a header injection attack.
  • Issued DLA 863-1 for the deluge BitTorrent client correcting a cross-site request forgery vulnerability.
  • Issued DLA 864-1 for jhead (an EXIF metadata tool) patching an arbitrary code execution vulnerability.
  • Issued DLA 865-1 for the suricata intrusion detection system, fixing an IP protocol matching error.
  • Issued DLA 871-1 for python3.2 fixing a TLS stripping vulnerability in the smptlib library.
  • Issued DLA 873-1 for apt-cacher preventing a HTTP response splitting vulnerability.
  • Issued DLA 876-1 for eject to prevent an issue regarding the checking of setuid(2) and setgid(2) return values.

Uploads
  • python-django:
    • 1:1.10.6-1 New upstream bugfix release.
    • 1:1.11~rc1-1 New upstream release candidate.
  • redis:
    • 3:3.2.8-2 Avoid conflict between RuntimeDirectory and tmpfiles.d(5) both attempting to create /run/redis with differing permissions. (#856116)
    • 3:3.2.8-3 Revert the creation of a /usr/bin/redis-check-rdb to /usr/bin/redis-server symlink to avoid a dangling symlink if only the redis-tools package is installed. (#858519)
  • gunicorn 19.7.0-1 & 19.7.1-1 New upstream releases.
  • adminer 4.3.0-1 New upstream release.

Finally, I also made the following non-maintainer uploads (NMUs):


FTP Team

As a Debian FTP assistant I ACCEPTed 121 packages: 4pane, adql, android-platform-system-core, android-sdk-helper, braillegraph, deepnano, dh-runit, django-auth-ldap, django-dirtyfields, drf-extensions, gammaray, gcc-7, gnome-keysign, golang-code.gitea-sdk, golang-github-bluebreezecf-opentsdb-goclient, golang-github-bsm-redeo, golang-github-cupcake-rdb, golang-github-denisenkom-go-mssqldb, golang-github-exponent-io-jsonpath, golang-github-facebookgo-ensure, golang-github-facebookgo-freeport, golang-github-facebookgo-grace, golang-github-facebookgo-httpdown, golang-github-facebookgo-stack, golang-github-facebookgo-subset, golang-github-go-openapi-loads, golang-github-go-openapi-runtime, golang-github-go-openapi-strfmt, golang-github-go-openapi-validate, golang-github-golang-geo, golang-github-gorilla-pat, golang-github-gorilla-securecookie, golang-github-issue9-assert, golang-github-issue9-identicon, golang-github-jaytaylor-html2text, golang-github-joho-godotenv, golang-github-juju-errors, golang-github-kisielk-gotool, golang-github-kubernetes-gengo, golang-github-lpabon-godbc, golang-github-lunny-log, golang-github-makenowjust-heredoc, golang-github-mrjones-oauth, golang-github-nbutton23-zxcvbn-go, golang-github-neelance-sourcemap, golang-github-ngaut-deadline, golang-github-ngaut-go-zookeeper, golang-github-ngaut-log, golang-github-ngaut-pools, golang-github-ngaut-sync2, golang-github-optiopay-kafka, golang-github-quobyte-api, golang-github-renstrom-dedent, golang-github-sergi-go-diff, golang-github-siddontang-go, golang-github-smartystreets-go-aws-auth, golang-github-xanzy-go-cloudstack, golang-github-xtaci-kcp, golang-github-yohcop-openid-go, graywolf, haskell-raaz, hfst-ospell, hikaricp, iptraf-ng, kanboard-cli, kcptun, kreport, libbluray, libcatmandu-store-elasticsearch-perl, libcsfml, libnet-prometheus-perl, libosmocore, libpandoc-wrapper-perl, libseqlib, matrix-synapse, mockldap, nfs-ganesha, node-buffer, node-pako, nose-el, nvptx-tools, nx-libs, open-ath9k-htc-firmware, pagein, paleomix, pgsql-ogr-fdw, profanity, pyosmium, python-biotools, python-django-extra-views, python-django-otp, python-django-push-notifications, python-dnslib, python-gmpy, python-gmpy2, python-holidays, python-kanboard, python-line-profiler, python-pgpy, python-pweave, python-raven, python-xapian-haystack, python-xopen, r-cran-v8, repetier-host, ruby-jar-dependencies, ruby-maven-libs, ruby-psych, ruby-retriable, seafile-client, spyder-unittest, stressant, systray-mdstat, telegram-desktop, thawab, tigris, tnseq-transit, typesafe-config, vibe.d, x2goserver & xmlrpc-c. I additionally filed 14 RC bugs against packages that had incomplete debian/copyright files against: golang-github-cupcake-rdb, golang-github-sergi-go-diff, graywolf, hfst-ospell, libbluray, pgsql-ogr-fdw, python-gmpy, python-gmpy2, python-pgpy, python-xapian-haystack, repetier-host, telegram-desktop, tigris & xmlrpc-c.

21 March 2017

Reproducible builds folks: Reproducible Builds: week 99 in Stretch cycle

Here's what happened in the Reproducible Builds effort between Sunday March 12 and Saturday March 18 2017: Upcoming events Reproducible Builds Hackathon Hamburg 2017 The Reproducible Builds Hamburg Hackathon 2017, or RB-HH-2017 for short is a 3 day hacking event taking place May 5th-7th in the CCC Hamburg Hackerspace located inside Frappant, as collective art space located in a historical monument in Hamburg, Germany. The aim of the hackathon is to spent some days working on Reproducible Builds in every distribution and project. The event is open to anybody interested on working on Reproducible Builds issues, with or without prior experience! Accomodation is available and travel sponsorship may be available by agreement. Please register your interest as soon as possible. Reproducible Builds Summit Berlin 2016 This is just a quick note, that all the pads we've written during the Berlin summit in December 2016 are now online (thanks to Holger), nicely complementing the report by Aspiration Tech. Request For Comments for new specification: BUILD_PATH_PREFIX_MAP Ximin Luo posted a draft version of our BUILD_PATH_PREFIX_MAP specification for passing build-time paths between high-level and low-level build tools. This is meant to help eliminate irreproducibility caused by different paths being used at build time. At the time of writing, this affects an estimated 15-20% of 25000 Debian packages. This is a continuation of an older proposal SOURCE_PREFIX_MAP, which has been updated based on feedback on our patches from GCC upstream, attendees of our Berlin 2016 summit, and participants on our mailing list. Thanks to everyone that contributed! The specification also contains runnable source code examples and test cases; see our git repo. Please comment on this draft ASAP - we plan to release version 1.0 of this in a few weeks. Toolchain changes Packages reviewed and fixed, and bugs filed Chris Lamb: Reviews of unreproducible packages 5 package reviews have been added, 274 have been updated and 800 have been removed in this week, adding to our knowledge about identified issues. 1 issue type has been added: Weekly QA work During our reproducibility testing, FTBFS bugs have been detected and reported by: diffoscope development diffoscope 79 and 80 were uploaded to experimental by Chris Lamb. It included contributions from: Chris Lamb: Maria Glukhova: strip-nondeterminism development strip-nondeterminism 0.032-1 was uploaded to unstable by Chris Lamb. It included contributions from: Chris Lamb: tests.reproducible-builds.org Misc. This week's edition was written by Ximin Luo, Holger Levsen and Chris Lamb & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

12 September 2016

Steve Kemp: If your code accepts URIs as input..

There are many online sites that accept reading input from remote locations. For example a site might try to extract all the text from a webpage, or show you the HTTP-headers a given server sends back in response to a request. If you run such a site you must make sure you validate the schema you're given - also remembering to do that if you're sent any HTTP-redirects.
Really the issue here is a confusion between URL & URI.
The only time I ever communicated with Aaron Swartz was unfortunately after his death, because I didn't make the connection. I randomly stumbled upon the html2text software he put together, which had an online demo containing a form for entering a location. I tried the obvious input:
file:///etc/passwd
The software was vulnerable, read the file, and showed it to me. The site gives errors on all inputs now, so it cannot be used to demonstrate the problem, but on Friday I saw another site on Hacker News with the very same input-issue, and it reminded me that there's a very real class of security problems here. The site in question was http://fuckyeahmarkdown.com/ and allows you to enter a URL to convert to markdown - I found this via the hacker news submission. The following link shows the contents of /etc/hosts, and demonstrates the problem: http://fuckyeahmarkdown.example.com/go/?u=file:///etc/hosts&read=1&preview=1&showframe=0&submit=go The output looked like this:
..
127.0.0.1 localhost
255.255.255.255 broadcasthost
::1 localhost
fe80::1%lo0 localhost
127.0.0.1 stage
127.0.0.1 files
127.0.0.1 brettt..
..
In the actual output of '/etc/passwd' all newlines had been stripped. (Which I now recognize as being an artifact of the markdown processing.) UPDATE: The problem is fixed now.

Reproducible builds folks: Reproducible Builds: week 72 in Stretch cycle

What happened in the Reproducible Builds effort between Sunday September 4 and Saturday September 10 2016: Reproducible work in other projects Python 3.6's dictonary type now retains the insertion order. Thanks to themill for the report. In coreboot, Alexander Couzens committed a change to make their release archives reproducible. Patches submitted Reviews of unreproducible packages We've been adding to our knowledge about identified issues. 3 issue types have been added: 1 issue type has been updated: 16 have been have updated: 13 have been removed, not including removed packages: 100s of packages have been tagged with the more generic captures_build_path, and many with captures_kernel_version, user_hostname_manually_added_requiring_further_investigation, user_hostname_manually_added_requiring_further_investigation, captures_shell_variable_in_autofoo_script, etc. Particular thanks to Emanuel Bronshtein for his work here. Weekly QA work FTBFS bugs have been reported by: diffoscope development strip-nondeterminism development tests.reproducible-builds.org: Misc. This week's edition was written by Chris Lamb and Holger Levsen and reviewed by a bunch of Reproducible Builds folks on IRC.

24 April 2016

Bits from Debian: Debian welcomes its 2016 summer interns

GSoC 2016 logo Outreachy logo We're excited to announce that Debian has selected 29 interns to work with us this summer: 4 in Outreachy, and 25 in the Google Summer of Code. Here is the list of projects and the interns who will work on them: Android SDK tools in Debian: APT - dpkg communications rework: Continuous Integration for Debian-Med packages: Extending the Debian Developer Horizon: Improving and extending AppRecommender: Improving the debsources frontend: Improving voice, video and chat communication with Free Software: MIPS and MIPSEL ports improvements: Reproducible Builds for Debian and Free Software: Support for KLEE in Debile: The Google Summer of Code and Outreachy programs are possible in Debian thanks to the effort of Debian developers and contributors that dedicate part of their free time to mentor students and outreach tasks. Join us and help extend Debian! You can follow the students weekly reports on the debian-outreach mailing-list, chat with us on our IRC channel or on each project's team mailing lists. Congratulations to all of them!

10 April 2016

Russ Allbery: Largish haul

Let's see if I can scrounge through all of my now-organized directories of ebooks and figure out what I haven't recorded here yet. At least the paper books make that relatively easy, since I don't shelve them until I post them. (Yeah, yeah, I should actually make a database.) Hugh Aldersey-Williams Periodic Tales (nonfiction)
Sandra Ulbrich Almazan SF Women A-Z (nonfiction)
Radley Balko Rise of the Warrior Cop (nonfiction)
Peter V. Brett The Warded Man (sff)
Lois McMaster Bujold Gentleman Jole and the Red Queen (sff)
Fred Clark The Anti-Christ Handbook Vol. 2 (nonfiction)
Dave Duncan West of January (sff)
Karl Fogel Producing Open Source Software (nonfiction)
Philip Gourevitch We Wish to Inform You That Tomorrow We Will Be Killed With Our Families (nonfiction)
Andrew Groen Empires of EVE (nonfiction)
John Harris @ Play (nonfiction)
David Hellman & Tevis Thompson Second Quest (graphic novel)
M.C.A. Hogarth Earthrise (sff)
S.L. Huang An Examination of Collegial Dynamics... (sff)
S.L. Huang & Kurt Hunt Up and Coming (sff anthology)
Kameron Hurley Infidel (sff)
Kevin Jackson-Mead & J. Robinson Wheeler IF Theory Reader (nonfiction)
Rosemary Kirstein The Lost Steersman (sff)
Rosemary Kirstein The Language of Power (sff)
Merritt Kopas Videogames for Humans (nonfiction)
Alisa Krasnostein & Alexandra Pierce (ed.) Letters to Tiptree (nonfiction)
Mathew Kumar Exp. Negatives (nonfiction)
Ken Liu The Grace of Kings (sff)
Susan MacGregor The Tattooed Witch (sff)
Helen Marshall Gifts for the One Who Comes After (sff collection)
Jack McDevitt Coming Home (sff)
Seanan McGuire A Red-Rose Chain (sff)
Seanan McGuire Velveteen vs. The Multiverse (sff)
Seanan McGuire The Winter Long (sff)
Marc Miller Agent of the Imperium (sff)
Randal Munroe Thing Explainer (graphic nonfiction)
Marguerite Reed Archangel (sff)
J.K. Rowling Harry Potter: The Complete Collection (sff)
K.J. Russell Tides of Possibility (sff anthology)
Robert J. Sawyer Starplex (sff)
Bruce Schneier Secrets & Lies (nonfiction)
Mike Selinker (ed.) The Kobold Game to Board Game Design (nonfiction)
Douglas Smith Chimerascope (sff collection)
Jonathan Strahan Fearsome Journeys (sff anthology)
Nick Suttner Shadow of the Colossus (nonfiction)
Aaron Swartz The Boy Who Could Change the World (essays)
Caitlin Sweet The Pattern Scars (sff)
John Szczepaniak The Untold History of Japanese Game Developers I (nonfiction)
John Szczepaniak The Untold History of Japanese Game Developers II (nonfiction)
Jeffrey Toobin The Run of His Life (nonfiction)
Hayden Trenholm Blood and Water (sff anthology)
Coen Teulings & Richard Baldwin (ed.) Secular Stagnation (nonfiction)
Ursula Vernon Book of the Wombat 2015 (graphic nonfiction)
Ursula Vernon Digger (graphic novel) Phew, that was a ton of stuff. A bunch of these were from two large StoryBundle bundles, which is a great source of cheap DRM-free ebooks, although still rather hit and miss. There's a lot of just fairly random stuff that's been accumulating for a while, even though I've not had a chance to read very much. Vacation upcoming, which will be a nice time to catch up on reading.

10 March 2016

Lunar: Reproducible builds: week 45 in Stretch cycle

What happened in the reproducible builds effort between February 28th and March 5th:

Toolchain fixes
  • Antonio Terceiro uploaded gem2deb/0.27 that forces generated gemspecs to use the date from debian/changelog.
  • Antonio Terceiro uploaded gem2deb/0.28 that forces generated gemspecs to have their contains file lists sorted.
  • Robert Luberda uploaded ispell/3.4.00-5 which make builds of hashes reproducible.
  • C dric Boutillier uploaded ruby-ronn/0.7.3-4 which will make the output locale agnostic. Original patch by Chris Lamb.
  • Markus Koschany uploaded spring/101.0+dfsg-1. Fixed by Alexandre Detiste.
Ximin Luo resubmitted the patch adding the --clamp-mtime option to Tar on Savannah's bug tracker. Lunar rebased our experimental dpkg on top of the current master branch. Changes in the test infrastructure are required before uploading a new version to our experimental repository. Reiner Herrmann rebased our custom texlive-bin against the latest uploaded version.

Packages fixed The following 77 packages have become reproducible due to changes in their build dependencies: asciidoctor, atig, fuel-astute, jekyll, libphone-ui-shr, linkchecker, maven-plugin-testing, node-iscroll, origami-pdf, plexus-digest, pry, python-avro, python-odf, rails, ruby-actionpack-xml-parser, ruby-active-model-serializers, ruby-activerecord-session-store, ruby-api-pagination, ruby-babosa, ruby-carrierwave, ruby-classifier-reborn, ruby-compass, ruby-concurrent, ruby-configurate, ruby-crack, ruby-css-parser, ruby-cucumber-rails, ruby-delorean, ruby-encryptor, ruby-fakeweb, ruby-flexmock, ruby-fog-vsphere, ruby-gemojione, ruby-git, ruby-grack, ruby-htmlentities, ruby-jekyll-feed, ruby-json-schema, ruby-listen, ruby-markerb, ruby-mathml, ruby-mini-magick, ruby-net-telnet, ruby-omniauth-azure-oauth2, ruby-omniauth-saml, ruby-org, ruby-origin, ruby-prawn, ruby-pygments.rb, ruby-raemon, ruby-rails-deprecated-sanitizer, ruby-raindrops, ruby-rbpdf, ruby-rbvmomi, ruby-recaptcha, ruby-ref, ruby-responders, ruby-rjb, ruby-rspec-rails, ruby-rspec, ruby-rufus-scheduler, ruby-sass-rails, ruby-sass, ruby-sentry-raven, ruby-sequel-pg, ruby-sequel, ruby-settingslogic, ruby-shoulda-matchers, ruby-slack-notifier, ruby-symboltable, ruby-timers, ruby-zip, ticgit, tmuxinator, vagrant, wagon, yard. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues, but not all of them: Patches submitted which have not made their way to the archive yet:
  • #816209 on elog by Reiner Herrmann: use printf instead of echo which is shell-independent.
  • #816214 on python-pip by Reiner Herrmann: removes timestamp from generated Python scripts.
  • #816230 on rows by Reiner Herrmann: tell grep to always treat the input as text.
  • #816232 on eficas by Reiner Herrmann: use printf instead of echo which is shell-independent.
Florent Daigniere and bancfc reported that linux-grsec was currently built with GRKERNSEC_RANDSTRUCT which will prevent reproducible builds with the current packaging.

tests.reproducible-builds.org pbuilder has been updated to the last version to be able to support Build-Depends-Arch and Build-Conflicts-Arch. (Mattia Rizzolo, h01ger) New package sets have been added for Subgraph OS, which is based on Debian Stretch: packages and build dependencies. (h01ger) Two new armhf build nodes have been added (thanks Vagrant Cascadian) and integrated in our Jenkins setup with 8 new armhf builder jobs. (h01ger)

strip-nondeterminism development strip-nondeterminism version 0.016-1 was released on Sunday 28th. It will now normalize the POT-Creation-Date field in GNU Gettext .mo files. (Reiner Herrmann) Several improvements to the packages metadata have also been made. (h01ger, Ben Finney)

Package reviews 185 reviews have been removed, 91 added and 33 updated in the previous week. New issue: fileorder_in_gemspec_files_list. 43 FTBFS bugs were reported by Chris Lamb, Martin Michlmayr, and gregor herrmann.

Misc. After merging the patch from Dhiru Kholia adding support for SOURCE_DATE_EPOCH in rpm, Florian Festi opened a discussion on the rpm-ecosystem mailing list about reproducible builds. On March 4th, Lunar gave an overview of the general reproducible builds effort at the Internet Freedom Festival in Valencia.

5 March 2016

Lunar: Reproducible builds: week 44 in Stretch cycle

What happened in the reproducible builds effort between February 21th and February 27th:

Toolchain fixes Didier Raboud uploaded pyppd/1.0.2-4 which makes PPD generation deterministic. Emmanuel Bourg uploaded plexus-maven-plugin/1.3.8-10 which sorts the components in the components.xml files generated by the plugin. Guillem Jover has implemented stable ordering for members of the control archives in .debs. Chris Lamb submitted another patch to improve reproducibility of files generated by cython.

Packages fixed The following packages have become reproducible due to changes in their build dependencies: dctrl-tools, debian-edu, dvdwizard, dymo-cups-drivers, ekg2, epson-inkjet-printer-escpr, expeyes, fades, foomatic-db, galternatives, gnuradio, gpodder, gutenprint icewm, invesalius, jodconverter-cli latex-mk, libiio, libimobiledevice, libmcrypt, libopendbx, lives, lttnganalyses, m2300w, microdc2, navit, po4a, ptouch-driver, pxljr, tasksel, tilda, vdr-plugin-infosatepg, xaos. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues, but not all of them:

tests.reproducible-builds.org The reproducibly tests for Debian now vary the provider of /bin/sh between bash and dash. (Reiner Herrmann)

diffoscope development diffoscope version 50 was released on February 27th. It adds a new comparator for PostScript files, makes the directory tests pass on slower hardware, and line ordering variations in .deb md5sums files will not be hidden anymore. Version 51 uploaded the next day re-added test data missing from the previous tarball. diffoscope is looking for a new primary maintainer.

Package reviews 87 reviews have been removed, 61 added and 43 updated in the previous week. New issues: captures_shell_variable_in_autofoo_script, varying_ordering_in_data_tar_gz_or_control_tar_gz. 30 new FTBFS have been reported by Chris Lamb, Antonio Terceiro, Aaron M. Ucko, Michael Tautschnig, and Tobias Frost.

Misc. The release team reported on their discussion about the topic of rebuilding all of Stretch to make it self-contained (in respect to reproducibility). Christian Boltz is hoping someone could talk about reproducible builds at the openSUSE conference happening June 22nd-26th in N rnberg, Germany.

4 February 2016

Daniel Pocock: Australians stuck abroad and alleged sex crimes

Two Australians have achieved prominence (or notoriety, depending on your perspective) for the difficulty in questioning them about their knowledge of alleged sex crimes. One is Julian Assange, holed up in the embassy of Ecuador in London. He is back in the news again today thanks to a UN panel finding that the UK is effectively detaining him, unlawfully, in the Ecuadorian embassy. The effort made to discredit and pursue Assange and other disruptive technologists, such as Aaron Swartz, has an eerie resemblance to the way the Inquisition hunted witches in the middle ages and beyond. The other Australian stuck abroad is Cardinal George Pell, the most senior figure in the Catholic Church in Australia. The Royal Commission into child sex abuse by priests has heard serious allegations claiming the Cardinal knew about and covered up abuse. This would appear far more sinister than anything Mr Assange is accused of. Like Mr Assange, the Cardinal has been unable to travel to attend questioning in person. News reports suggest he is ill and can't leave Rome, although he is being accommodated in significantly more comfort than Mr Assange. If you had to choose, which would you prefer to leave your child alone with?

5 January 2016

Benjamin Mako Hill: Celebrate Aaron Swartz in Seattle (or Atlanta, Chicago, Dallas, NYC, SF)

I m organizing an event at the University of Washington in Seattle that involves a reading, the screening of a documentary film, and a Q&A about Aaron Swartz. The event coincides with the third anniversary of Aaron s death and the release of a new book of Swartz s writing that I contributed to. aaronsw-tiob_bwcstw The event is free and open the public and details are below:

WHEN: Wednesday, January 13 at 6:30-9:30 p.m.

WHERE: Communications Building (CMU) 120, University of Washington

We invite you to celebrate the life and activism efforts of Aaron Swartz, hosted by UW Communication professor Benjamin Mako Hill. The event is next week and will consist of a short book reading, a screening of a documentary about Aaron s life, and a Q&A with Mako who knew Aaron well details are below. No RSVP required; we hope you can join us.

Aaron Swartz was a programming prodigy, entrepreneur, and information activist who contributed to the core Internet protocol RSS and co-founded Reddit, among other groundbreaking work. However, it was his efforts in social justice and political organizing combined with his aggressive approach to promoting increased access to information that entangled him in a two-year legal nightmare that ended with the taking of his own life at the age of 26.

January 11, 2016 marks the third anniversary of his death. Join us two days later for a reading from a new posthumous collection of Swartz s writing published by New Press, a showing of The Internet s Own Boy (a documentary about his life), and a Q&A with UW Communication professor Benjamin Mako Hill a former roommate and friend of Swartz and a contributor to and co-editor of the first section of the new book. If you re not in Seattle, there are events with similar programs being organized in Atlanta, Chicago, Dallas, New York, and San Francisco. All of these other events will be on Monday January 11 and registration is required for all of them. I will be speaking at the event in San Francisco.

4 January 2016

Lunar: Reproducible builds: week 36 in Stretch cycle

What happened in the reproducible builds effort between December 27th and January 2nd: Infrastructure dak now silently accepts and discards .buildinfo files (commit 1, 2), thanks to Niels Thykier and Ansgar Burchardt. This was later confirmed as working by Mattia Rizzolo. Packages fixed The following packages have become reproducible due to changes in their build dependencies: banshee-community-extensions, javamail, mono-debugger-libs, python-avro. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues, but not all of them: Untested changes: reproducible.debian.net The testing distribution (the upcoming stretch) is now tested on armhf. (h01ger) Four new armhf build nodes provided by Vagrant Cascandian were integrated in the infrastructer. This allowed for 9 new armhf builder jobs. (h01ger) The RPM-based build system, koji, is now in unstable and testing. (Marek Marczykowski-G recki, Ximin Luo). Package reviews 131 reviews have been removed, 71 added and 53 updated in the previous week. 58 new FTBFS reports were made by Chris Lamb and Chris West. New issues identified this week: nondeterminstic_ordering_in_gsettings_glib_enums_xml, nondeterminstic_output_in_warnings_generated_by_breathe, qt_translate_noop_nondeterminstic_ordering. Misc. Steven Chamberlain explained in length why reproducible cross-building across architectures mattered, and posted results of his tests comparing a stage1 debootstrapped chroot of linux-i386 once done from official Debian packages, the others cross-built from kfreebsd-amd64.

Benjamin Mako Hill: The Boy Who Could Change the World: The Writings of Aaron Swartz

The New Press has published a new collection of Aaron Swartz s writing called The Boy Who Could Change the World: The Writings of Aaron Swartz. I worked with Seth Schoen to introduce and help edit the opening section of book that includes Aaron s writings on free culture, access to information and knowledge, and copyright. Seth and I have put our introduction online under an appropriately free license (CC BY-SA). aaronsw_book_coverOver the last week, I ve read the whole book again. I think the book really is a wonderful snapshot of Aaron s thought and personality. It s got bits that make me roll my eyes, bits that make me want to shout in support, and bits that continue to challenge me. It all makes me miss Aaron terribly. I strongly recommend the book. Because the publication is post-humous, it s meant that folks like me are doing media work for the book. In honor of naming the book their progressive pick of the week, Truthout has also published an interview with me about Aaron and the book. Other folks who introduced and/or edited topical sections in the book are David Auerbach (Computers), David Segal (Politics), Cory Doctorow (Media), James Grimmelmann (Books and Culture), and Astra Taylor (Unschool). The book is introduced by Larry Lessig.

2 January 2016

Daniel Pocock: The great life of Ian Murdock and police brutality in context

Tributes: (You can Follow or Tweet about this blog on Twitter) Over the last week, people have been saying a lot about the wonderful life of Ian Murdock and his contributions to Debian and the world of free software. According to one news site, a San Francisco police officer, Grace Gatpandan, has been doing the opposite, starting a PR spin operation, leaking snippets of information about what may have happened during Ian's final 24 hours. Sadly, these things are now starting to be regurgitated without proper scrutiny by the mainstream press (note the erroneous reference to SFGate with link to SFBay.ca, this is British tabloid media at its best). The report talks about somebody (no suggestion that it was even Ian) "trying to break into a residence". Let's translate that from the spin-doctor-speak back to English: it is the silly season, when many people have a couple of extra drinks and do silly things like losing their keys. "a residence", or just their own home perhaps? Maybe some AirBNB guest arriving late to the irritation of annoyed neighbours? Doesn't the choice of words make the motive sound so much more sinister? Nobody knows the full story and nobody knows if this was Ian, so snippets of information like this are inappropriate, especially when somebody is deceased. Did they really mean to leave people with the impression that one of the greatest visionaries of the Linux world was also a cat burglar? That somebody who spent his life giving selflessly and generously for the benefit of the whole world (his legacy is far greater than Steve Jobs, as Debian comes with no strings attached) spends the Christmas weekend taking things from other people's houses in the dark of the night? The report doesn't mention any evidence of a break-in or any charges for breaking-in. If having a few drinks and losing your keys in December is such a sorry state to be in, many of us could potentially be framed in the same terms at some point in our lives. That is one of the reasons I feel so compelled to write this: somebody else could be going through exactly the same experience at the moment you are reading this. Any of us could end up facing an assault as unpleasant as the tweets imply at some point in the future. At least I can console myself that as a privileged white male, the risk to myself is much lower than for those with mental illness, the homeless, transgender, Muslim or black people but as the tweets suggest, it could be any of us. The story reports that officers didn't actually come across Ian breaking in to anything, they encountered him at a nearby street corner. If he had weapons or drugs or he was known to police that would have almost certainly been emphasized. Is it right to rush in and deprive somebody of their liberties without first giving them an opportunity to identify themselves and possibly confirm if they had a reason to be there? The report goes on, "he was belligerent", "he became violent", "banging his head" all by himself. How often do you see intelligent and successful people like Ian Murdock spontaneously harming themselves in that way? Can you find anything like that in any of the 4,390 Ian Murdock videos on YouTube? How much more frequently do you see reports that somebody "banged their head", all by themselves of course, during some encounter with law enforcement? Do police never make mistakes like other human beings? If any person was genuinely trying to spontaneously inflict a head injury on himself, as the police have suggested, why wouldn't the police leave them in the hospital or other suitable care? Do they really think that when people are displaying signs of self-harm, rounding them up and taking them to jail will be in their best interests? Now, I'm not suggesting this started out with some sort of conspiracy. Police may have been at the end of a long shift (and it is a disgrace that many US police are not paid for their overtime) or just had a rough experience with somebody far more sinister. On the other hand, there may have been a mistake, gaps in police training or an inappropriate use of a procedure that is not always justified, like a strip search, that causes profound suffering for many victims. A select number of US police forces have been shamed around the world for a series of incidents of extreme violence in recent times, including the death of Michael Brown in Ferguson, shooting Walter Scott in the back, death of Freddie Gray in Baltimore and the attempts of Chicago's police to run an on-shore version of Guantanamo Bay. Beyond those highly violent incidents, the world has also seen the abuse of Ahmed Mohamed, the Muslim schoolboy arrested for his interest in electronics and in 2013, the suicide of Aaron Swartz which appears to be a direct consequence of the "Justice" department's obsession with him. What have the police learned from all this bad publicity? Are they changing their methods, or just hiring more spin doctors? If that is their response, then doesn't it leave them with a cruel advantage over those people who were deceased? Isn't it standard practice for some police to simply round up anybody who is a bit lost and write up a charge sheet for resisting arrest or assaulting an officer as insurance against questions about their own excessive use of force? When British police executed Jean Charles de Menezes on a crowded tube train and realized they had just done something incredibly outrageous, their PR office went to great lengths to try and protect their image, even photoshopping images of Menezes to make him look more like some other suspect in a wanted poster. To this day, they continue to refer to Menezes as a victim of the terrorists, could they be any more arrogant? While nobody believes the police woke up that morning thinking "let's kill some random guy on the tube", it is clear they made a mistake and like many people (not just police), they immediately prioritized protecting their reputation over protecting the truth. Nobody else knows exactly what Ian was doing and exactly what the police did to him. We may never know. However, any disparaging or irrelevant comments from the police should be viewed with some caution. The horrors of incarceration It would be hard for any of us to understand everything that an innocent person goes through when detained by the police. The recently released movie about The Stanford Prison Experiment may be an interesting place to start, a German version produced in 2001, Das Experiment, is also very highly respected. The United States has the largest prison population in the world and the second-highest per-capita incarceration rate. Many, including some on death row, are actually innocent, in the wrong place at the wrong time, without the funds to hire an attorney. The system, and the police and prison officers who operate it, treat these people as packages on a conveyor belt, without even the most basic human dignity. Whether their encounter lasts for just a few hours or decades, is it any surprise that something dies inside them when they discover this cruel side of American society? Worldwide, there is an increasing trend to make incarceration as degrading as possible. People may be innocent until proven guilty, but this hasn't stopped police in the UK from locking up and strip-searching over 4,500 children in a five year period, would these children go away feeling any different than if they had an encounter with Jimmy Saville or Rolf Harris? One can only wonder what they do to adults. What all this boils down to is that people shouldn't really be incarcerated unless it is clear the danger they pose to society is greater than the danger they may face in a prison. What can people do for Ian and for justice? Now that these unfortunate smears have appeared, it would be great to try and fill the Internet with stories of the great things Ian has done for the world. Write whatever you feel about Ian's work and your own experience of Debian. While the circumstances of the final tweets from his Twitter account are confusing, the tweets appear to be consistent with many other complaints about US law enforcement. Are there positive things that people can do in their community to help reduce the harm? Sending books to prisoners (the UK tried to ban this) can make a difference. Treat them like humans, even if the system doesn't. Recording incidents of police activities can also make a huge difference, such as the video of the shooting of Walter Scott or the UK police making a brutal unprovoked attack on a newspaper vendor. Don't just walk past a situation and assume everything is under control. People making recordings may find themselves in danger, it is recommended to use software that automatically duplicates each recording, preferably to the cloud, so that if the police ask you to delete such evidence, you can let them watch you delete it and still have a copy. Can anybody think of awards that Ian Murdock should be nominated for, either in free software, computing or engineering in general? Some, like the prestigious Queen Elizabeth Prize for Engineering can't be awarded posthumously but others may be within reach. Come and share your ideas on the debian-project mailing list, there are already some here. Best of all, Ian didn't just build software, he built an organization, Debian. Debian's principles have helped to unite many people from otherwise different backgrounds and carry on those principles even when Ian is no longer among us. Find out more, install it on your computer or even look for ways to participate in the project.

Lunar: Reproducible builds: week 33 in Stretch cycle

What happened in the reproducible builds effort between December 6th and December 12th: Toolchain fixes Reiner Herrmann rebased our experimental version of doxygen on version 1.8.9.1-6. Chris Lamb submitted a patch to make the manpages generated by ruby-ronn reproducible by using the locale-agnostic %Y-%m-%d for the dates. Daniel Kahn Gillmor took another shot at the issue of source path captured in DWARF symbols. A patch has been sent for review by GCC upstream to add the ability to read an environment variable with -fdebug-prefix-map. Packages fixed The following 24 packages have become reproducible due to changes in their build dependencies: gkeyfile-sharp, gprbuild, graphmonkey, gthumb, haskell-yi-language, ion, jackson-databind, jackson-dataformat-smile, jackson-dataformat-xml, jnr-ffi, libcommons-net-java, libproxy, maven-shared-utils, monodevelop-database, mydumper, ndesk-dbus, nini, notify-sharp, pixz, protozero, python-rtslib-fb, slurm-llnl, taglib-sharp, tomboy-latex. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues, but not all of them: These uploads might have fixed reproducibility issues but could not be tested yet: Patches submitted which have not made their way to the archive yet: reproducible.debian.net Files created with diffoscope now have diffoscope in their name instead debbindiff. (h01ger) Hostnames of first and second build node are now recorded and shown in the build history. (Mattia Rizzolo) Exchanges have started with F-Droid developers to better understand what would be required to test F-Droid applications. (h01ger) A first small set of Fedora 23 packages is now also being tested while development on a new framework for testing RPMs in general has begun. A new Jenkins job has been added to set up to mock, the build system used by Fedora. Another new job takes care of testing RPMs from Fedora 23 on x86_64. So far only 151 packages from the buildsys-build group are tested (currently all unreproducible), but the plan is to build all 17,000 source packages in Fedora 23 and rawhide. The page presenting the results should also soon be improved. (h01ger, Dhiru Kholia) For Arch Linux, all 2223 packages from the extra repository will also be tested from now on. Packages in extra" are tested every four weeks, while those from core every week. Statistics are now displayed alongside the results. (h01ger) jenkins.debian.net has been updated to jenkins-job-builder version 1.3.0. Many job configurations have been simplified and refactored using features of the new version. This was another milestone for the jenkins.debian.org migration. (Phil Hands, h01ger) diffoscope development Chris Lamb announced try.diffoscope.org: an online service that runs diffoscope on user provided files. Screenshot of try.diffoscope.org Improvements are welcome. The application is licensed under the AGPLv3. On diffoscope itself, most pending patches have now been merged. Expect a release soon! Most of the code implementing parallel processing has been polished. Sadly, unpacking archive is CPU-bound in most cases, so the current thread-only implementation does not offer much gain on big packages. More work is still require to also add concurrent processes. Documentation update Ximin Luo has started to write a specification for buildinfo files that could become a larger platform than the limited set of features that were thought so far for Debian .buildinfo. Package reviews 113 reviews have been removed, 111 added and 56 updated in the previous week. 42 new FTBFS bugs were opened by Chris Lamb and Niko Tyni. New issues identified this week: timestamps_in_documentation_generated_by_docbook_dbtimestamp, timestamps_in_sym_l_files_generated_by_malaga, timestamps_in_edj_files_generated_by_edje_cc. Misc. Chris Lamb presented reproducible builds at skroutz.gr.

7 December 2015

Francois Marier: Tweaking Cookies For Privacy in Firefox

Cookies are an important part of the Web since they are the primary mechanism that websites use to maintain user sessions. Unfortunately, they are also abused by surveillance marketing companies to follow you around the Web. Here are a few things you can do in Firefox to protect your privacy.

Cookie Expiry Cookies are sent from the website to your browser via a Set-Cookie HTTP header on the response. It looks like this:
HTTP/1.1 200 OK
Date: Mon, 07 Dec 2015 16:55:43 GMT
Server: Apache
Set-Cookie: SESSIONID=65576c6c64206e6f2c657920756f632061726b636465742065686320646f2165
Content-Length: 2036
Content-Type: text/html;charset=UTF-8
When your browser sees this, it saves that cookie for the given hostname and keeps it until you close the browser. Should a site want to persist their cookie for longer, they can add an Expires attribute:
Set-Cookie: SESSIONID=65576c...; expires=Tue, 06-Dec-2016 22:38:26 GMT
in which case the browser will retain the cookie until the server-provided expiry date (which could be in a few years). Of course, that's if you don't instruct your browser to do things differently.

Third-Party Cookies So far, we've only looked at first-party cookies: the ones set by the website you visit and which are typically used to synchronize your login state with the server. There is however another kind: third-party cookies. These ones are set by the third-party resources that a page loads. For example, if a page loads JavaScript from a third-party ad network, you can be pretty confident that they will set their own cookie in order to build a profile on you and serve you "better and more relevant ads".

Controlling Third-Party Cookies If you'd like to opt out of these, you have a couple of options. The first one is to turn off third-party cookies entirely by going back into the Privacy preferences and selecting "Never" next to the "Accept third-party cookies" setting (network.cookie.cookieBehavior = 1). Unfortunately, turning off third-party cookies entirely tends to break a number of sites which rely on this functionality (for example as part of their for login process). A more forgiving option is to accept third-party cookies only for sites which you have actually visited directly. For example, if you visit Facebook and login, you will get a cookie from them. Then when you visit other sites which include Facebook widgets they will not recognize you unless you allow cookies to be sent in a third-party context. To do that, choose the "From visited" option (network.cookie.cookieBehavior = 3). In addition to this setting, you can also choose to make all third-party cookies automatically expire when you close Firefox by setting the network.cookie.thirdparty.sessionOnly option to true in about:config.

Other Ways to Limit Third-Party Cookies Another way to limit undesirable third-party cookies is to tell the browser to avoid connecting to trackers in the first place. This functionality is now built into Private Browsing mode and enabled by default. To enable it outside of Private Browsing too, simply go into about:config and set privacy.trackingprotection.enabled to true. You could also install the EFF's Privacy Badger add-on which uses heuristics to detect and block trackers, unlike Firefox tracking protection which uses a blocklist of known trackers.

My Recommended Settings On my work computer I currently use the following:
network.cookie.cookieBehavior = 3
network.cookie.lifetimePolicy = 3
network.cookie.lifetime.days = 5
network.cookie.thirdparty.sessionOnly = true
privacy.trackingprotection.enabled = true
which allows me to stay logged into most sites for the whole week (no matter now often I restart Firefox Nightly) while limiting tracking and other undesirable cookies as much as possible.

14 November 2015

Juliana Louback: PaperTrail - Powered by IBM Watson

On the final semester of my MSc program at Columbia SEAS, I was lucky enough to be able to attend a seminar course taught by Alfio Gliozzo entitled Q&A with IBM Watson. A significant part of the course is dedicated to learning how to leverage the services and resources available on the Watson Developer Cloud. This post describes the course project my team developed, the PaperTrail application.

Project Proposal Create an application to assist in the development of future academic papers. Based on a paper s initial proposal, Paper Trail predicts publications to be used as references or acknowledgement of prior art and provides a trend analysis of major topics and methods. The objective is to speed the discovery of relevant papers early in the research process, and allow for early assessment of the depth of prior research concerning the initial proposal.

Meet the Team Wesley Bruning, Software Engineer, MSc. in Computer Science Xavier Gonzalez, Industrial Engineer, MSc. in Data Science Juliana Louback, Software Engineer, MSc. in Computer Science Aaron Zakem, Patent Attorney, MSc. in Computer Science

Prior Art A significant amount of attention has been given to this topic over the past few decades. The table below shows the work the team deemed most relevant due to recency, accuracy and similarity of functionality. priorArt The variation in accuracy displayed is a result of experimentation with different dataset sizes and algorithm variations. More information and details can be found in the prior art report. The main differential of PaperTrail is providing a form of access to the citation prediciton and trend analysis algorithm. With the exception of the project by McNee et al., these algorithmns aren t currently available for general use. The application on researchindex.net is open to use but its objective is to rank publications and authors for given topics.

Algorithm Citation Prediction: PaperTrail builds on the work done by Wolski s team in Fall 2014. This algorithmn builds a reference graph used to define research communities, with an associated vector of topic scores generated by an LDA model. The papers in each research community are then ranked by importance within the community with a custom ranking algorithm. When a target document is given to algorithm as input, the LDA model is used to generate a vector of topics that are present in the document. The communities with the most similar topic vectors are selected and the publications within these communities with highest rank and greatest similarity to the input document are recommended as references. A more detailed description can be found here. Trend Analysis: Initially, the idea was to use the AlchemyData News API to obtain statistics pertaining to the amount of publications on a given topic over time. However, with the exception of buzz-words (i.e. big data ), many more specialized topics appeared very infrequently in news articles, if at all. This isn t entirely surprising given the target audience of PaperTrail. As a work around, we use the Alchemy Language API to extract keywords from the abstracts in the dataset, in addition to relevance scores. The PaperTrail database could then be queried for entry counts for a given year and keyword to provide an indication of publication trends in academia. Note that the Alchemy Language API extracts multiple-word keywords as well as single words.

Data To maintain consistency with Wolski s project, we are using the DBLP data as made available on aminer.org. The DBLP-Citation-network V5 dataset contains 1,572,277 entries; we are limited to the use of entries that contain both abstracts and citations, bringing the dataset size down to 265,865 entries.

Architecture A high-level visualization of the project architecture is displayed below. Before launching PaperTrail, it s necessary to train Wolski s algorithm offline. Currently any documentation with regard to the performance of said algorithm is unavailable; the PaperTrail project will include an evaluation phase and report the findings made. The PaperTrail app and database will be hosted on the Bluemix Platform. ptArchitecture

Status Report Phases completed:
  • Project design
  • Prior art research
  • Data cleansing
  • Development and deployment of an alpha version of the PaperTrail app
Phases under development:
  • Algorithm training and evaluation
  • Keyword extraction
  • MapReduce of publication frequency by year and topic
  • Data visualization component

7 November 2015

Mehdi Dogguy: 3rd annual Aaron Swartz Day, November 7-8

This weekend is organized the Aaron Swartz Day across the world. There are events organized in many cities and video streams available. It is important that we remember Aaron's projects and fights. If you want to know more about Aaron Swartz, you may start by watching the excellent documentary The Internet's Own Boy : The Story of Aaron Swartz. His work was very inspirational and should not be forgotten!

8 October 2015

Petter Reinholdtsen: The Story of Aaron Swartz - Let us all weep!

The movie "The Internet's Own Boy: The Story of Aaron Swartz" is both inspiring and depressing at the same time. The work of Aaron Swartz has inspired me in my work, and I am grateful of all the improvements he was able to initiate or complete. I wish I am able to do as much good in my life as he did in his. Every minute of this 1:45 long movie is inspiring in documenting how much impact a single person can have on improving the society and this world. And it is depressing in documenting how the law enforcement of USA (and other countries) is corrupted to a point where they can push a bright kid to his death for downloading too many scientific articles. Aaron is dead. Let us all weep. The movie is also available on Youtube. I wish there were Norwegian subtitles available, so I could show it to my parents.

Petter Reinholdtsen: The Story of Aron Swartz - Let us all weep!

The movie "The Internet's Own Boy: The Story of Aaron Swartz" is both inspiring and depressing at the same time. The work of Aaron Swartz has inspired me in my work, and I am grateful of all the improvements he was able to initiate or complete. I wish I am able to do as much good in my life as he did in his. Every minute of this 1:45 long movie is inspiring in documenting how much impact a single person can have on improving the society and this world. And it is depressing in documenting how the law enforcement of USA (and other countries) is corrupted to a point where they can push a bright kid to his death for downloading too many scientific articles. Aron is dead. Let us all weep. The movie is also available on Youtube. I wish there were Norwegian subtitles available, so I could show it to my parents.

27 July 2015

Michael Stapelberg: dh-make-golang: creating Debian packages from Go packages

Recently, the pkg-go team has been quite busy, uploading dozens of Go library packages in order to be able to package gcsfuse (a user-space file system for interacting with Google Cloud Storage) and InfluxDB (an open-source distributed time series database). Packaging Go library packages (!) is a fairly repetitive process, so before starting my work on the dependencies for gcsfuse, I started writing a tool called dh-make-golang. Just like dh-make itself, the goal is to automatically create (almost) an entire Debian package. As I worked my way through the dependencies of gcsfuse, I refined how the tool works, and now I believe it s good enough for a first release. To demonstrate how the tool works, let s assume we want to package the Go library github.com/jacobsa/ratelimit:
midna /tmp $ dh-make-golang github.com/jacobsa/ratelimit
2015/07/25 18:25:39 Downloading "github.com/jacobsa/ratelimit/..."
2015/07/25 18:25:53 Determining upstream version number
2015/07/25 18:25:53 Package version is "0.0~git20150723.0.2ca5e0c"
2015/07/25 18:25:53 Determining dependencies
2015/07/25 18:25:55 
2015/07/25 18:25:55 Packaging successfully created in /tmp/golang-github-jacobsa-ratelimit
2015/07/25 18:25:55 
2015/07/25 18:25:55 Resolve all TODOs in itp-golang-github-jacobsa-ratelimit.txt, then email it out:
2015/07/25 18:25:55     sendmail -t -f < itp-golang-github-jacobsa-ratelimit.txt
2015/07/25 18:25:55 
2015/07/25 18:25:55 Resolve all the TODOs in debian/, find them using:
2015/07/25 18:25:55     grep -r TODO debian
2015/07/25 18:25:55 
2015/07/25 18:25:55 To build the package, commit the packaging and use gbp buildpackage:
2015/07/25 18:25:55     git add debian && git commit -a -m 'Initial packaging'
2015/07/25 18:25:55     gbp buildpackage --git-pbuilder
2015/07/25 18:25:55 
2015/07/25 18:25:55 To create the packaging git repository on alioth, use:
2015/07/25 18:25:55     ssh git.debian.org "/git/pkg-go/setup-repository golang-github-jacobsa-ratelimit 'Packaging for golang-github-jacobsa-ratelimit'"
2015/07/25 18:25:55 
2015/07/25 18:25:55 Once you are happy with your packaging, push it to alioth using:
2015/07/25 18:25:55     git push git+ssh://git.debian.org/git/pkg-go/packages/golang-github-jacobsa-ratelimit.git --tags master pristine-tar upstream
The ITP is often the most labor-intensive part of the packaging process, because any number of auto-detected values might be wrong: the repository owner might not be the Upstream Author , the repository might not have a short description, the long description might need some adjustments or the license might not be auto-detected.
midna /tmp $ cat itp-golang-github-jacobsa-ratelimit.txt
From: "Michael Stapelberg" <stapelberg AT debian.org>
To: submit@bugs.debian.org
Subject: ITP: golang-github-jacobsa-ratelimit -- Go package for rate limiting
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Package: wnpp
Severity: wishlist
Owner: Michael Stapelberg <stapelberg AT debian.org>
* Package name    : golang-github-jacobsa-ratelimit
  Version         : 0.0~git20150723.0.2ca5e0c-1
  Upstream Author : Aaron Jacobs
* URL             : https://github.com/jacobsa/ratelimit
* License         : Apache-2.0
  Programming Lang: Go
  Description     : Go package for rate limiting
 GoDoc (https://godoc.org/github.com/jacobsa/ratelimit)
 .
 This package contains code for dealing with rate limiting. See the
 reference (http://godoc.org/github.com/jacobsa/ratelimit) for more info.
TODO: perhaps reasoning
midna /tmp $
After filling in all the TODOs in the file, let s mail it out and get a sense of what else still needs to be done:
midna /tmp $ sendmail -t -f < itp-golang-github-jacobsa-ratelimit.txt
midna /tmp $ cd golang-github-jacobsa-ratelimit
midna /tmp/golang-github-jacobsa-ratelimit master $ grep -r TODO debian
debian/changelog:  * Initial release (Closes: TODO) 
midna /tmp/golang-github-jacobsa-ratelimit master $
After filling in these TODOs as well, let s have a final look at what we re about to build:
midna /tmp/golang-github-jacobsa-ratelimit master $ head -100 debian/**/*
==> debian/changelog <==                            
golang-github-jacobsa-ratelimit (0.0~git20150723.0.2ca5e0c-1) unstable; urgency=medium
  * Initial release (Closes: #793646)
 -- Michael Stapelberg <stapelberg@debian.org>  Sat, 25 Jul 2015 23:26:34 +0200
==> debian/compat <==
9
==> debian/control <==
Source: golang-github-jacobsa-ratelimit
Section: devel
Priority: extra
Maintainer: pkg-go <pkg-go-maintainers@lists.alioth.debian.org>
Uploaders: Michael Stapelberg <stapelberg@debian.org>
Build-Depends: debhelper (>= 9),
               dh-golang,
               golang-go,
               golang-github-jacobsa-gcloud-dev,
               golang-github-jacobsa-oglematchers-dev,
               golang-github-jacobsa-ogletest-dev,
               golang-github-jacobsa-syncutil-dev,
               golang-golang-x-net-dev
Standards-Version: 3.9.6
Homepage: https://github.com/jacobsa/ratelimit
Vcs-Browser: http://anonscm.debian.org/gitweb/?p=pkg-go/packages/golang-github-jacobsa-ratelimit.git;a=summary
Vcs-Git: git://anonscm.debian.org/pkg-go/packages/golang-github-jacobsa-ratelimit.git
Package: golang-github-jacobsa-ratelimit-dev
Architecture: all
Depends: $ shlibs:Depends ,
         $ misc:Depends ,
         golang-go,
         golang-github-jacobsa-gcloud-dev,
         golang-github-jacobsa-oglematchers-dev,
         golang-github-jacobsa-ogletest-dev,
         golang-github-jacobsa-syncutil-dev,
         golang-golang-x-net-dev
Built-Using: $ misc:Built-Using 
Description: Go package for rate limiting
 This package contains code for dealing with rate limiting. See the
 reference (http://godoc.org/github.com/jacobsa/ratelimit) for more info.
==> debian/copyright <==
Format: http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Upstream-Name: ratelimit
Source: https://github.com/jacobsa/ratelimit
Files: *
Copyright: 2015 Aaron Jacobs
License: Apache-2.0
Files: debian/*
Copyright: 2015 Michael Stapelberg <stapelberg@debian.org>
License: Apache-2.0
Comment: Debian packaging is licensed under the same terms as upstream
License: Apache-2.0
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
 You may obtain a copy of the License at
 .
 http://www.apache.org/licenses/LICENSE-2.0
 .
 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
 .
 On Debian systems, the complete text of the Apache version 2.0 license
 can be found in "/usr/share/common-licenses/Apache-2.0".
==> debian/gbp.conf <==
[DEFAULT]
pristine-tar = True
==> debian/rules <==
#!/usr/bin/make -f
export DH_GOPKG := github.com/jacobsa/ratelimit
%:
	dh $@ --buildsystem=golang --with=golang
==> debian/source <==
head: error reading  debian/source : Is a directory
==> debian/source/format <==
3.0 (quilt)
midna /tmp/golang-github-jacobsa-ratelimit master $
Okay, then. Let s give it a shot and see if it builds:
midna /tmp/golang-github-jacobsa-ratelimit master $ git add debian && git commit -a -m 'Initial packaging'
[master 48f4c25] Initial packaging                                                      
 7 files changed, 75 insertions(+)
 create mode 100644 debian/changelog
 create mode 100644 debian/compat
 create mode 100644 debian/control
 create mode 100644 debian/copyright
 create mode 100644 debian/gbp.conf
 create mode 100755 debian/rules
 create mode 100644 debian/source/format
midna /tmp/golang-github-jacobsa-ratelimit master $ gbp buildpackage --git-pbuilder
[ ]
midna /tmp/golang-github-jacobsa-ratelimit master $ lintian ../golang-github-jacobsa-ratelimit_0.0\~git20150723.0.2ca5e0c-1_amd64.changes
I: golang-github-jacobsa-ratelimit source: debian-watch-file-is-missing
P: golang-github-jacobsa-ratelimit-dev: no-upstream-changelog
I: golang-github-jacobsa-ratelimit-dev: extended-description-is-probably-too-short
midna /tmp/golang-github-jacobsa-ratelimit master $
This package just built (as it should!), but occasionally one might need to disable a test and file an upstream bug about it. So, let s push this package to pkg-go and upload it:
midna /tmp/golang-github-jacobsa-ratelimit master $ ssh git.debian.org "/git/pkg-go/setup-repository golang-github-jacobsa-ratelimit 'Packaging for golang-github-jacobsa-ratelimit'"
Initialized empty shared Git repository in /srv/git.debian.org/git/pkg-go/packages/golang-github-jacobsa-ratelimit.git/
HEAD is now at ea6b1c5 add mrconfig for dh-make-golang
[master c5be5a1] add mrconfig for golang-github-jacobsa-ratelimit
 1 file changed, 3 insertions(+)
To /git/pkg-go/meta.git
   ea6b1c5..c5be5a1  master -> master
midna /tmp/golang-github-jacobsa-ratelimit master $ git push git+ssh://git.debian.org/git/pkg-go/packages/golang-github-jacobsa-ratelimit.git --tags master pristine-tar upstream
Counting objects: 31, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (25/25), done.
Writing objects: 100% (31/31), 18.38 KiB   0 bytes/s, done.
Total 31 (delta 2), reused 0 (delta 0)
To git+ssh://git.debian.org/git/pkg-go/packages/golang-github-jacobsa-ratelimit.git
 * [new branch]      master -> master
 * [new branch]      pristine-tar -> pristine-tar
 * [new branch]      upstream -> upstream
 * [new tag]         upstream/0.0_git20150723.0.2ca5e0c -> upstream/0.0_git20150723.0.2ca5e0c
midna /tmp/golang-github-jacobsa-ratelimit master $ cd ..
midna /tmp $ debsign golang-github-jacobsa-ratelimit_0.0\~git20150723.0.2ca5e0c-1_amd64.changes
[ ]
midna /tmp $ dput golang-github-jacobsa-ratelimit_0.0\~git20150723.0.2ca5e0c-1_amd64.changes   
Uploading golang-github-jacobsa-ratelimit using ftp to ftp-master (host: ftp.upload.debian.org; directory: /pub/UploadQueue/)
[ ]
Uploading golang-github-jacobsa-ratelimit_0.0~git20150723.0.2ca5e0c-1.dsc
Uploading golang-github-jacobsa-ratelimit_0.0~git20150723.0.2ca5e0c.orig.tar.bz2
Uploading golang-github-jacobsa-ratelimit_0.0~git20150723.0.2ca5e0c-1.debian.tar.xz
Uploading golang-github-jacobsa-ratelimit-dev_0.0~git20150723.0.2ca5e0c-1_all.deb
Uploading golang-github-jacobsa-ratelimit_0.0~git20150723.0.2ca5e0c-1_amd64.changes
midna /tmp $ cd golang-github-jacobsa-ratelimit 
midna /tmp/golang-github-jacobsa-ratelimit master $ git tag debian/0.0_git20150723.0.2ca5e0c-1
midna /tmp/golang-github-jacobsa-ratelimit master $ git push git+ssh://git.debian.org/git/pkg-go/packages/golang-github-jacobsa-ratelimit.git --tags master pristine-tar upstream
Total 0 (delta 0), reused 0 (delta 0)
To git+ssh://git.debian.org/git/pkg-go/packages/golang-github-jacobsa-ratelimit.git
 * [new tag]         debian/0.0_git20150723.0.2ca5e0c-1 -> debian/0.0_git20150723.0.2ca5e0c-1
midna /tmp/golang-github-jacobsa-ratelimit master $
Thanks for reading this far, and I hope dh-make-golang makes your life a tiny bit easier. As dh-make-golang just entered Debian unstable, you can install it using apt-get install dh-make-golang. If you have any feedback, I m eager to hear it.

Next.

Previous.